In a typical application server or web server environment, where there exists perhaps many hundreds or thousands of concurrent users (clients), each of these clients can send requests to the server. FIG. 1 shows an example of such an environment. A modern server 100 may have many processors, and many threads 102 servicing those processors. The threads take requests 110, 112, 114 from the clients 104, 106, 108, and apply them to the processors in a queue-like fashion. The actual number of threads used has a big impact on the performance of the server, and its ability to handle large numbers of users. However, figuring out the optimal number of threads is a complicated process.
One approach to this problem has been to add even more threads, so that the number of threads greatly exceeds the number of processors. This ensures that requests are almost immediately fed into threads, but doesn't ensure that those requests are quickly taken up by the processors. A more recent alternative has been to create thread pools, with a flexible number of threads operating within. The actual number of threads can then be fine-tuned by an administrator to provide the best performance for a particular runtime environment. However, this process is heavily dependent on the skill of the administrator, and, since it is largely environment-dependent and static, is of most use only for test-benching and marketing purposes. In real-life situations, the environments are so varied that this static approach is not satisfactory.
Accordingly, what is needed is a means for allowing a server to automatically determine and implement the optimal number of concurrent threads based on objective predictions for the workload. Because of the limitations of where an application server sits (above the operating system) this thread control cannot be embedded within the operating system, but must be able to exist at a higher level. Accordingly there should be a means for controlling the number of the threads in the application server, and the queues that feed them.